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i.  Introduction 
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Purpose  of  this  presentation 


•This  presentation  tells... 

•Project  overview  in  SEMPR 
•Benchmark  planning  parameters  in  SEMPR 

•Benchmark  project  level  performance  and  work  item  (component)  level 
performance 
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About  SEMPR 


•Software  Engineering  Measured  and  Performance  Repository 


•SEI  has  collected  data  from  organizations  that  have  adopted  TSP  in 
SEMPR 


•Stores  project  data  in  Tuma  Solutions  Team  Process  Data  Warehouse 


•  From  109  project  cycles  (in  this  report) 

•  Used  the  Software  Process  Dashboard 
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How  did  we  measure  data  quality  in  SEMPR 


•Time  log  and  defect  log  have  high  correctness  and  consistency  by 
automatic  data  recording. 

•Size  log  and  task  log  have  low  correctness  by  manual  data  recording. 
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What  do  the  data  tell  us? 

2.  SEMPR  data  analysis 
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How  are  projects  organized? 
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How  many  projects  are  found  in  each  pattern? 
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What  were  the  project  durations? 


histogram  of  duration  in  parent  project 
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What  are  the  planning  period  durations? 
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How  many  team  members  on  projects? 


histogram  of  team  size  in  project 
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How  many  task  hours  per  week? 

mean  Team  Member  Weekly  Direct  Hours  per  Week 


Frequency 


Mean  Team  Member  Weekly  Task  Hours 
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actual  task  hours 


How  do  Plan  and  Actual  planned  project  hours  compare? 


the  scatterplot:  Plan  task  hours  vs.  Actual  task  hours 

in  project 


Project  level  time  hours  data 
is  high  predictable. 


the  scatterplot:  Log  plan  task  hours  vs.  Log  actual  task  hours 
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actual  task  hours 


How  do  Plan  and  Actual  component  hours  compare? 

(work  item) 


the  scatterplot:  Plan  task  hours  vs.  Actual  task  hours 
in  component 


Log  transformed  work  item  level 
time  hours  data  is  predictable. 


the  scatterplot:  Log  plan  task  hours  vs.  Log  actual  task  hours 

in  component 
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Frequency 


How  did  they  perform  to  planned  schedule? 

- 1— — ; - r  \  schedule  performance  = 

|  Project  performance  | 


histogram  of  schedule  performance  in  all  projects 
overflow  bin  =  5 


actual  duration  -  plan  duration 
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[Work  item  performance) 

histogram  of  schedule  performance  in  all  work  items 
overflow  bin  =  2 
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Frequency 


How  well  did  they  estimate  effort? 

_  effort  performance  = 


[  Project  performance  1 


histogram  of  effort  performance  in  all  projects 
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plan  effort 
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histogram  of  effort  performance  in  all  work  items 
overflow  bin  =  2 
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How  fast  are  defects  injectioned? 
(all  work  items)  * 


:=- 

o 


<Xi 


O' 

O 


histogram  of  defect  injection  rate  in  all  work  items 
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What  were  the  defect  removal 


(all  work  items) 
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histogram  of  defect  removal  rate  in  all  work  items 
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How  did  defect  injection  rates  differ  by  phase 


|  All  phase 

boxplot  of  defect  injection  rate  in  all  work  items 


DIR  in  code  review  has  wide 
range  and  highest  median. 


|  Except  code  phase 

boxplot  of  defect  injection  rate  in  all  work  items 
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How  did  defect  removal  rates  differ  by  phase 
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What  were  the  total  defect  densities 


histogram  of  defect  density  in  all  work  items 
overflow  bin  =  300 
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Agenda 


3.  Conclusion 


Software  Engineering  Institute 


Carnegie  Mellon  University 


TSP  Symposium  2014 
William  Nichols  Nov  4,  2014 

©2014  Carnegie  Mellon  University 


Conclusion 


SEMPR  collects  TSP  project  data  for  benchmark  and  analysis 
Projects  organize  in  many  ways 
Benchmarks  include 

•  distributions  for  defect  injection  and  removal  rates 

•  Ranges  of  task  hours 

•  Effort  estimation  accuracy 

•  Schedule  estimation  accuracy 

Much  work  remains 

•  Include  more  contextual  data 

•  Continue  to  add  projects  the  database 
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